Many approaches have been developed for encoding machine readable information on objects. Perhaps the most well-known technique is the ubiquitous barcode. Over the years, various barcode types have been created, which differ significantly from the name-sake pattern of dark linear bars on a lighter background. Now, the term barcode has been used to encompass machine symbols in various shapes, sizes, patterns and colors.
These types of codes were primarily developed to be reliable data carriers that could be applied with a wide array of print and marking techniques on many types of objects. They were not designed, in most cases, to be aesthetically pleasing, or to be weaved into other imagery, whether it be graphical designs, text etc. on product packaging, labels, or displays. As such, in most applications, these codes occupy a dedicated location on an object, where no other information is located. This approach has worked well to reliably apply identifying information to objects, including packaging, parts (“Direct Part Marking”), etc. Nevertheless, placing the code at a dedicated location limits the ability to find and read the code. When used on consumer goods, it is often located on the back or bottom of the item so as not to interfere with consumer messaging and product aesthetics. This placement of the code tends to slow down code reading or increase scanner cost by requiring additional components to capture multiple views of an object. It also increases risk of injury due to the need for workers to re-position the object to scan the code for auto-identification. Obviously, this undermines the theoretical speed and reliability advantages of auto-identification.
Data signaling techniques have been developed that have promise in addressing these drawbacks while providing additional advantages. One such technique is referred to as digital watermarking. Digital watermarking provides a method of encoding information within image content or object surface topology. As such, it can be applied over some or all of the surface of an object with minimal cost and changes to workflow, addressing the drawbacks of barcodes while being fully compatible with them. Additionally, the digital watermarking applies to many different types of media, including analog and digital forms of images (including video) and audio. This enables enterprises to implement auto-identification and auxiliary data encoding across all of their assets, including physical and digital media. Some (but not all) embodiments of digital watermarking include hiding the presence of the signal.
Digital watermarking often carries information. This information is often referred to as the “watermark payload” or “watermark message,” a variable sequence of message symbols inserted per unit of host content. The watermark message can carry variable information. In one example, the watermark message carries information corresponding to a so-called Global Trade Item Number (GTIN). A GTIN is an identifier for trade items developed by GS1. Such identifiers are used to look up product information in a database (often by entering the number through a bar code scanner pointed at an actual product) which may belong to a retailer, manufacturer, collector, researcher, or other entity.
There are, of course, limits to the extent to which a watermark payload can be inserted in existing image or audio (the host signal) without impacting the perceptual quality of the host. Generally speaking, host signals with more variable information provide greater opportunity to insert the payload, whereas host signals with uniform or solid tones provide less opportunity for insertion of the payload. In cases where there is little host signal content, it may not be possible to encode the payload, or if it is encoded, it is done so with a greater impact on perceptual quality.
Perceptual quality relates to the extent to which a human perceives a change in an image. This is a challenging metric, as it has a significant subjective component. Nevertheless, the human visual system has been modeled, and data from user tests can be used to construct a classifier that measures in a quantitative way, whether a change to an image or video will be visible or deemed objectionable. Human visual system models and classifiers based on them provide a measure of whether a change made to an image is likely to be visible, and also, can quantify visibility in units such as Just Noticeable Difference (JND) units. For applications where images are modified to insert a data signal, the perceptual quality is a constraint on data signal design and encoding strategy. The importance of this constraint on signal design and encoding methodology varies with the application. The flexibility of the watermark signal allows it to be transformed into visual elements of the object's design in various ways, and as such, provides many options for adapting the signal to satisfy perceptual quality constraints for different types of images and applications.
Literature documenting our earlier work describes various ways to deal with host signal types that lack signal content compatible with data encoding. We refer to one approach as “sparse” marking as the data carrying signal is formed as a sparse array of signal elements. For visual media, the sparse array of elements works well on portions of a host image that are uniform or solid tones or appear largely blank. With greater sophistication in the signaling, it also is effective in encoding blank areas around text of a document, label, visual display or package, as our signaling schemes employ robust data encoding strategies to mitigate impact of interference from the text. In one embodiment, a sparse mark is comprised of a pattern of spatial locations where ink is deposited or not. For example, the sparse signal may be comprised of ink dots on a light background, such that the signal forms a pattern of subtly darker spatial locations. The signal is designed to be sparse by the spacing apart of the darker locations on the light background. Conversely, the signal may be designed as an array of lighter “holes” on a relatively darker background. See, for example, U.S. Pat. Nos. 6,345,104, 6,993,152 and 7,340,076, which are hereby incorporated by reference in their entirety.
As described in U.S. Pat. No. 6,345,104, this strategy of forming patterns of light and dark elements is consistent with our earlier digital watermarking strategies that modulate luminance. For example, a lighter element encodes a first message symbol (e.g., binary one), while a darker element encodes a second symbol (e.g., binary zero).
The sparse signal has minimal impact on visual quality due to its sparse arrangement. However, the trade-off for applications like automatic object identification is that more sophistication is required in the data signaling methodology to ensure that the data carried within the sparse signal may be reliably and efficiently recovered in many different and challenging environments. The sparse nature of the signal dictates that less payload may be encoded per unit of object surface area. Further, within the sparse signal, there is a trade-off between allocating signal for payload capacity versus signal for robustness. In the latter category of robustness, the signaling scheme must support recovery in environments of geometric distortion, which occurs when the sparse signal is imaged from various angles, perspectives and distances, in the presence of noise of various types that tends to interfere with the data signal.
There are various sources of geometric distortion that need to be addressed to reliably recover the payload in the sparse signal. Examples of geometric distortion include signal cropping and warping. Cropping truncates portions of the sparse signal, e.g., in cases where only a portion is captured due to occlusion by other objects or incomplete capture by a scanner. Warping occurs when the surface on which the sparse signal is applied is curved (on cups or cans) or wrinkled (on bags and flexible plastic or foil pouches) and when the sparse signal is imaged from a surface at various perspectives.
The design of a signaling scheme must also account for practical challenges posed by constraints on digital circuitry, processors and memory for encoding and decoding. These include computational efficiency, power consumption, memory consumption, memory bandwidth, use of network bandwidth, cost of hardware circuitry or programmable processors/circuitry, cost of designing and integrating encoders and decoders within signal transmitter and receiver, equipment, etc. For example, some encoding schemes may provide optimized encoding or decoding, but may not be applicable because they are too slow for encoding or decoding in real time, e.g., as the host signal is being transmitted, received, updated, or being processed with multiple other signal processing operations concurrently.
The design must also account for practical challenges of the marking technology. The printing technology must be able to reproduce the signal reliably. This includes transformation of the data signal in the Raster Image Processor as well as application of an image to an object.
The design must also account for practical challenges posed by 2D image capture and associated optics. Scanners at Point of Sale (POS), for example, tend to be tuned to detect black and white barcodes (e.g., with a spectral range that focuses on image capture around image content at a wavelength at or around 660 nm), and as such, the illumination type and sensors may have a much more limited range of spectral bands and resolutions that the device can sense, e.g., in the range of 630-710 nm. When we use the term “at or around 660 nm,” we mean a wavelength within the range of 640-680 nm. In one particular example, a red illumination scanner includes a wavelength in the range of 650-670 nm, and may be centered or include a peak in this range, e.g., at 660 nm. In another example, a red illumination scanner includes a wavelength in the range of 650-710 nm, which we refer to as “at or around 690 nm”. Other scanners may be centered or include a peak, e.g., at 688-690 nm.
Sparse signaling is particularly challenging in that the sparse nature of the signal provides less opportunity to include signal for payload and robustness. In particular, there is less opportunity to include payload and synchronization. The strategy for synchronization may rely on an explicit synchronization component or an implicit synchronization component. In the latter case, the encoding of the payload may be arranged in a manner that provides a pattern that facilitates detection and synchronization.
Another important consideration for some applications is compatibility and inter-operability with other messaging protocols. For example, in the case of applying identification over the surface of objects, the signal encoding and decoding strategy should preferably support various protocols to deal with various image types, printing technologies, and scanner technologies. This design consideration dictates that sparse signaling should be compatible with encoding and decoding other signaling, like legacy encoding schemes on older objects and dense watermark signaling strategies and barcodes of various types. Preferably, the installed base of decoder technology should be able to efficiently decode signals from various signaling types, including new sparse signal arrangements.
One aspect of the disclosure is a method for inserting a sparse, variable data carrying signal into an image. This method provides a first signal component, which facilitates a synchronization function of the sparse signal. It also provides a second signal component that is modulated to carry a variable data signal. The first and second signal components have values located at coordinates within a two-dimensional block. The method combines the first signal component and the second signal component to produce the sparse, variable data carrying signal by setting sparse elements at coordinates within the two-dimensional block where the first and second component signal values provide compatible modulation of the image. The method inserts the sparse, variable data carrying signal into at least a first image layer or channel of an image design.
In this method, the sparse signal elements are set at coordinates where the signal components provide compatible modulation. Compatible modulation is where the signals of the two components comply with consistency rule. For example, the signals have a consistent direction of modulation of an optical property at a coordinate within the two-dimensional block. One optical property is brightness or lightness and it may be modulated in a darker or lighter direction. Other examples of optical properties are chrominance and hue. Components that modulate an image in the same direction, namely both darker or both lighter, have a compatible modulation direction.
The consistency rule may also be determined by consistency in amount of modulation. For instance, modulation of the components is compatible if the value of signal components fall within a common range bounded by at least one threshold. In some cases, components are multi-valued, and then quantized to quantization levels or bins (multi-dimensional values falling in a bin space are quantized to a bin). This may be implemented by applying a threshold, in which values within a range are set to a particular value. Multi-valued components are converted into binary levels (e.g., 1 or 0) or more than two levels (e.g., 1, 0, −1), for example. The consistency rule is evaluated on the output of this process by comparing the values of the signal components at a coordinate and setting sparse element or not at the coordinate based on the consistency. A logical AND operation or like comparison may be used to determine whether the signal components satisfy the consistency rule, e.g., quantize to a common quantization level or range bounded by a threshold.
The quantizing of a signal can be used to determine a distribution of sparse elements in a block. In one embodiment, for example, a signal component (e.g., sync component) is quantized to provide a sparse distribution within a block. The sparse signal is then formed based on where the signal components comply with a consistency rule. The consistency rule evaluates signal components at these locations and sets a sparse element at the coordinate where they are consistent.
One target-rich environment for applying a sparse mark (also referred to as a “sparse watermark” or “sparse digital watermark”) is consumer packaged goods (e.g., boxes of cereal, yogurt packages, cans of soup, etc.). However, some packaging materials and printing processes present challenges. For example, so-called dry offset printing (discussed below) does not accommodate overprinting. Flexography printing creates other problems in keeping tight printing tolerances. Other challenges include watermarking light (or dark) printed areas under the spectral constraints of most POS scanners (e.g., red LED scanners).
We address some of these problems by conveying a digital watermark (e.g., a sparse or dense mark) with narrow-band absorption (and other) materials. Such materials can be combined with a coating such as a varnish, laminate, ink extender, clear primer, adhesive, sealant or other coating. The coating can be provided over tricky color areas without adversely affecting the aesthetics of a consumer packaged good design.
Another aspect of this disclosure is a printed package for a retail product comprising a substrate. The substrate includes a first area and a second area, in which the first area and the second area comprise adjacent non-overlapping areas. The first area comprises a first offset ink or flexography ink printed thereon. The printed package comprises a coating, the coating comprising a narrow-band absorption material with a spectral absorbance peak in the range of 630 nm-710 nm. The coating is selectively applied over portions of both the first area and the second area in a 2D pattern representing an encoded digital watermark signal, the 2D pattern being redundantly applied on the printed package. The printed package comprises more area within the first area and the second area without the coating than area with the coating. The one or more instances of the 2D pattern are detectable from machine-analysis of illumination of the printed package, the illumination have an illumination peak in the range of 630 nm-710 nm.
A related aspect of the disclosure is a system comprising the above package, a point of sale scanner comprising an LED with an illumination peak in the range of 630 nm-710 nm, and a digital watermark detector for analyzing 2D imagery captured by the point of sale scanner to recover the encoded digital watermark signal.
Yet another aspect of the disclosure is a printed package for a retail product prepared by a process comprising: providing a sparse pattern of elements at coordinates within a two-dimensional block, providing a signal component, the signal component being modulated to carry a variable data signal, and generating a sparse, variable data carrying signal by distributing the signal component within a two-dimensional block based on the location of the sparse pattern of elements; applying a first ink on a surface of the printed package, the surface comprising a first area and a second area, the first ink applied only in the first area; applying a coating over portions of both the first area and the second area according to a pattern representing the two-dimensional block, the coating comprising a narrow-band absorption material with a spectral absorption having an absorbance peak in the range of 630 nm-710 nm, the two-dimensional block being redundantly applied on the printed package, in which the printed package comprises more area within the first and second areas without the coating than area with the coating, in which one or more instances of the two-dimensional block are detectable from machine-analysis of illumination of the printed package, the illumination have an illumination peak in the range of 630 nm-710 nm.
Another aspect of this disclosure provides a method comprising: scanning a printed object with a red illumination scanner, the red illumination scanner having a wavelength at or around 660 nm, said scanning yielding scan data, wherein the printed object includes a coating printed thereon through an offset printing press, the coating including a narrow-band absorption additive that absorbs at or around 660 nm, the coating printed in a manner to convey an encoded plural-bit message, the encoded plural-bit message corresponding to a GTIN number; analyzing the scan data with one or more programmed processors to decode the encoded plural bit message, said analyzing yielding the GTIN number; and providing the GTIN number as an output.
Yet another aspect of the disclosure provides a method of offsetting color casting for a printed object. The method includes: providing a first additive that absorbs light energy at or around a center frequency of an illumination source; providing a second additive that absorbs in the ultra-violet spectrum, yet fluoresces at or around the center frequency of the illumination source, wherein a combination of spectral responses of the first additive and the second additive offset color casting; printing the first additive, second additive and a color on the printed object, wherein the printing conveys an encoded plural bit signal.
In this document, we detail various signaling schemes, including schemes for generating signals, and encoding and decoding them in various object types and object marking technologies. We describe schemes that encode a sparse signal within host signal carrier in a manner that is robust, flexible to achieve perceptual quality constraints, and provides improved data capacity. We also describe use of narrow-band absorption materials which may correspond with a central wavelength and width of a narrow-band LED (or laser) illumination, e.g., from a Point of Sale (POS) scanner.